Interpreting Blackbox Models via Model Extraction

ABSTRACT The ability to interpret machine learning models has become incredibly important as machine learning is increasingly used to inform consequential decisions. We propose an approach to interpreting complex, blackbox models by constructing global explanations that summarize their reasoning process. In our approach, a global explanation is a decision tree that approximates the blackbox model. As long as the decision tree is a good approximation, then the reasoning process of the decision tree mirrors that of the blackbox model. We devise a novel algorithm for extracting decision tree explanations that actively samples new training points to avoid overfitting. We evaluate our algorithm on a random forest to predict diabetes risk, and a learned control policy for the cart-pole problem. Compared to several baselines, the decision trees extracted by our algorithm are substantially more accurate and are equally or more interpretable based on a user study. Finally, we describe several insights we derived based on our interpretations, including a causal issue that we validated with a physician.